The Asymptotic Behaviour of a General Finite Nonhomogeneous Markov Chain (the Decomposition-separation Theorem)
نویسنده
چکیده
The Decomposition-Separation Theorem generalizing the classical KolmogorovDoeblin results about the decomposition of finite homogeneous Markov chains to the nonhomogeneous case is presented. The ground-breaking result in this direction was given in the work of David Blackwell in 1945. The relation of this theorem with other problems in probability theory and Markov Decision Processes is discussed. Dedicated to David Blackwell in deep respect for his many wonderful mathematical
منابع مشابه
The Decomposition-Separation Theorem for Finite Nonhomogeneous Markov Chains and Related Problems
Let M be a finite set, P be a stochastic matrix and U = {(Zn)} be the family of all finite Markov chains (MC) (Zn) defined by M, P , and all possible initial distributions. The behavior of a MC (Zn) is a classical result of Probability Theory derived in the 30’s by A. Kolmogorov and W. Doeblin. If a stochastic matrix P is replaced by a sequence of stochastic matrices (Pn) and transitions at mom...
متن کاملOn $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes
In the present paper we investigate the $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes with general state spaces. We provide a necessary and sufficient condition for such processes to satisfy the $L_1$-weak ergodicity. Moreover, we apply the obtained results to establish $L_1$-weak ergodicity of quadratic stochastic processes.
متن کاملA Central Limit Theorem for Temporally Nonhomogenous Markov Chains with Applications to Dynamic Programming
We prove a central limit theorem for a class of additive processes that arise naturally in the theory of finite horizon Markov decision problems. The main theorem generalizes a classic result of Dobrushin for temporally nonhomogeneous Markov chains, and the principal innovation is that here the summands are permitted to depend on both the current state and a bounded number of future states of t...
متن کاملProducts of Stochastic Matrices and Applications
This paper deals with aspects of the limit behaviour of products of nonidentical finite or countable stochastic matrices (P). Applications n are given to nonhomogeneous Markov models as positive chains, some classes of finite chains considered by Doeblin and weakly ergodic chains.
متن کاملRelative Entropy Rate between a Markov Chain and Its Corresponding Hidden Markov Chain
In this paper we study the relative entropy rate between a homogeneous Markov chain and a hidden Markov chain defined by observing the output of a discrete stochastic channel whose input is the finite state space homogeneous stationary Markov chain. For this purpose, we obtain the relative entropy between two finite subsequences of above mentioned chains with the help of the definition of...
متن کامل